[fix][broker] Replication stuck when partitions count between two clusters is not the same#22983
Conversation
|
When a topic type is different(non-partitioned/partitioned) on the local and remote clusters, the pulsar-client has been thrown |
|
@poorbarcode You mentioned "break-change" in the PR's title. Do you mean this PR will introduce break change or it fixed a break change? If it will fix a break change, could you please link the change which introduced the break change? |
Sorry, I assumed that the code class cast was added by #21946, if so, I checked the history commits, and the class cast code was added when creating the class So there is no break change( all the two scenarios can not work ), I removed the label |
Yes, it does. see here if (metadata.partitions != 0) {
log.error("[{}] The partitions count between two clusters is not the same(remote partitions: {}),"
+ " the replicator can not be created successfully: {}", replicatorId, metadata.partitions,
state);
// This exception will be caught below, so it can be any typed.
checkPartitionsSameFuture.completeExceptionally(new RuntimeException(replicatorId
+ "Can not replicate data to a partitioned topic."));
}This PR only changes the scenario that there is no topic on the remote cluster. And makes the error logs clearer |
|
/pulsarbot rerun-failure-checks |
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/OneWayReplicatorTest.java
Outdated
Show resolved
Hide resolved
pulsar-client/src/main/java/org/apache/pulsar/client/impl/conf/ProducerConfigurationData.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractReplicator.java
Outdated
Show resolved
Hide resolved
… two clusters is not the same
7143776 to
1ceee52
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #22983 +/- ##
============================================
- Coverage 73.57% 73.45% -0.13%
+ Complexity 32624 2412 -30212
============================================
Files 1877 1914 +37
Lines 139502 143616 +4114
Branches 15299 15668 +369
============================================
+ Hits 102638 105489 +2851
- Misses 28908 30049 +1141
- Partials 7956 8078 +122
Flags with carried forward coverage won't be shown. Click here to find out more.
|
|
Since |
…sters is not the same (apache#22983) (cherry picked from commit a8ce990) (cherry picked from commit 25542d8)
(cherry picked from commit fdd9747)
…sters is not the same (apache#22983) (cherry picked from commit a8ce990) (cherry picked from commit 25542d8)
(cherry picked from commit fdd9747)
…sters is not the same (apache#22983)
…sters is not the same (apache#22983) (cherry picked from commit a8ce990) Signed-off-by: Zixuan Liu <nodeces@gmail.com>
…sters is not the same (apache#22983) (cherry picked from commit a8ce990) Signed-off-by: Zixuan Liu <nodeces@gmail.com>
…sters is not the same (apache#22983) (cherry picked from commit a8ce990) Signed-off-by: Zixuan Liu <nodeces@gmail.com>
…sters is not the same (apache#22983) (cherry picked from commit a8ce990) Signed-off-by: Zixuan Liu <nodeces@gmail.com>
Motivation
Auto-creation rule of both clusters
cluster-c1:non-partitionedcluster-c2:partitionedtopic with2partitionsIssue 1 (Issue)
c1 -> c2&c2 -> c1.tp1onc1.c2, but it will fail due to aClassCaseExceptionpic-1non-partitionedtopictp-1partitionedtopictp-1with2partitions.tp-1-partition-0andtp-1-partition-1on the clusterc2will also start replication.c1c1:non-partitionedtopictp-1non-partitionedtopictp-1-partition-0non-partitionedtopictp-1-partition-1c2:partitionedtopictp-1with2partitions.tp-1-partition-0andtp-1-partition-1on the clusterc1will also start replication.non-partitionedtopictp-1is already connected to the remote cluster, the new topics' replicator can not be created successfully and it will get an errorProducer with name 'pulsar.repl.c1--> c2' is already connected to topic tp-1-partition-0/tp-1-partition-1Issue 2
c1 -> c2.tp1onc1.c2, but it will fail due to aClassCaseExceptionpic-1c1:non-partitionedtopictp-1c2:partitionedtopictp-1with2partitions.pic-1:

Modifications
Replicator will only trigger a non-partitioned topic creation.
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: x